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Abstract 

We consider boolean circuits computing n-operators / : {0, 1}" — >■ 
{0, 1}". As gates we allow arbitrary boolean functions; neither fanin nor 
fanout of gates is restricted. An operator is linear if it computes n linear 
forms, that is, computes a matrix- vector product Ax over GF(2). 

We prove the existence of n-operators requiring about wires in any 
circuit, and linear n-operators requiring about n^/logn wires in depth- 
2 circuits, if either all output gates or all gates on the middle layer are 
linear. 

1 Introduction 

We consider general circuits computing n-operators / : {0,1}" — ^ {0,1}". As 
gates we allow arbitrary boolean functions of their inputs; there is no restriction 
on their fanin or fanout. Thus, the phenomenon which causes complexity of 
such circuits is information transfer rather than information processing as in 
the case of single functions. Such a circuit is a directed acyclic graph with n 
input nodes a;i,...,x„ and n output nodes t/i,...,y„. Each non-input node 
computes some boolean function of its predecessors. A circuit computes an 
operator / — (/i, . . . , /„) if, for alH = 1, . . . , n, the boolean function computed 
at the ith output node is the ith component fi of the operator /. The depth 
of a circuit is the largest number of wires in a path from an input to an output 
node. 

The size of a circuit is the total number of wires in it. We will denote by Sd{f) 
the smallest number of wires in a general circuit of depth at most d computing /. 
If there are no restrictions on the depth, the corresponding measure is denoted 
by s(/). Note that s{f) < si(/) < ri^ holds for any n-operator, so quadratic 
lower bounds are the highest ones. 

Circuits of depth 2 constitute the first non-trivial model. Interest in depth-2 
circuits comes from the following important result of Valiant jl7j : If in every 
depth-2 circuit, computing / with 0(n/ In Inn) gates on the middle layer, at 
least n^"'"^*-"'^-' direct wires must connect inputs with output gates, then / cannot 
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be computed by log-depth circuits with a hnear number of fanin-2 gates. To 
prove a super-hnear lower bound for log-depth circuits is an old and well-known 
problem in circuit complexity. 

Super-linear lower bounds up to S2(/) = fl{n\og^ n) where proved using 
graph-theoretic arguments by analyzing some super-concentration properties of 
the circuit as a graph [51[ni[ini[Il[ni[Il[Tl[Il[in]. Higher lower bounds of 
the form S2(/) = i^{n^^^) were recently proved using information theoretical 
arguments [H |5] . For larger depth d known lower bounds are only slightly non- 
linear. All these bounds, however, are on the total number of wires, so they still 
have no consequences for log-depth circuits. 

In fact, in the class of general circuits, even the question about the com- 
plexity of a random operator remained unclear. In particular, it was unclear 
whether operators requiring a quadratic number of wires (even in depth 2) exist 



2 Circuits for general operators 

Note that a direct counting argument, as in the case of constant fanin circuits, 
does not work for general circuits: already for d > n + log n, the number 2^ 
of possible boolean functions that may be assigned to a node of fanin d may be 
larger than the total number 2"^ of n-operators. 

Our first result is an observation that this bad situation can be excluded 
by just turning the power of circuits against themselves to ensure that, in an 
optimal circuit, no gate can have fanin larger than n. This leads us to 

Theorem 1. For almost all n-operators f, s{f) — il{n'^). 

Proof. Let fJ,{L) be the number of different n-operators computable by boolean 
circuits with at most L wires. Our goal is to upper bound this number in terms 
of n and L, and compare this bound with the total number 2"^ of n-operators. 

Take an optimal circuit with i < L wires computing some n-operator; hence, 
£ < n"^. Then £ = J2^i where di, . . . , are the fanins of its gates. It is 
clear that we need m > n gates, since we must have n input gates. On the other 
hand, m<£ + n + 2< 1r? gates are always enough since every non-input gate, 
besides two possible constant gates, must have nonzero fanin. 

We now make use of the fact that the gates in our circuits may be arbitrary 
boolean functions: This allows us to assume that di < n for all i. Indeed, if 
di > n, then we can replace the zth gate by the boolean function computed at 
this gate and join it to all n input variables; when doing this, the total number 
of wires in the circuit can only decrease. 

The number of sequences di , . . . , d^^ of fanins with < di < n does not 
exceed (n + 1)™. For each such sequence and for each i — 1, . . . , m, there are at 
most (™) < rn''* possibilities to chose the set of inputs for the ith node and at 
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most 2^ ' possibilities to assign a boolean function to this node. Hence, 

m m 

fi{L) <{n + 1)" n m"^- n 2"'' = (" + 1)™"^^^-^ *2^" i . 
Since dt < L <in? and m < 2n^, this yields 

m 

log2/.(L)<^2'^'+0(n2 log2 n). 

We now observe that at most n/2 nodes can have fanin larger than 2L/n, for 
otherwise we would have more than {2L/n) ■ {n/2) = L wires in total. Since 
m < 2tt? and since the fanin of each gate does not exceed n, we obtain that 

m 

2"^' < (m - n/2)22^/" + (n/2)2" < 2^24^/" + n2"-i . 

i=l 

Hence, 

log2 ^i{L) < 2n^4^/" + n2"-i + 0{n^ log^ n) . (1) 

Since the total nmnber of operators / : {0, 1}" — > {0, 1}" is 2"^", the smallest 
number L of wires sufhcient to compute all of them must satisfy log2 fJ-{L) > n2". 
By ([TJ, this implies 

2n24^/" > n2"-i - 0{n^ log^ n) . 
Dividing both sides by 2n^, we obtain that 4^/" = f^(2"/n), and hence, L = 



3 Circuits for linear operators 

An important class of operators are linear ones. Each such operator computes n 
linear forms, that is, computes a matrix- vector product fA{x) — Ax over GF{2) 
where A is an n x n (0, l)-matrix. We are interested in the complexity S2(/a) 
of such operators in the class of depth- 2 circuits. 

If all gates are required to be linear (parities and their negations), then easy 
counting shows that some linear operators require r2(n^/logn) wires. It is also 
known that 0{n^ / \ogn) are also sufficient to compute any linear operator [161 

But what if we allow arbitrary (non-linear) boolean functions as gates — can 
we then compute linear operators J a more efficiently? The largest known lower 
bound for an explicit linear operator J a has the form S2(/a) ~ ^l{n\ogn) 
This raises the following question: Do linear n-operators requiring S2(/a) = 
J7(n^/logn) wires exist at all? We are only able to answer this question pos- 
itively under the additional restriction that either all output gates of all gates 
on the middle layer must be linear functions. 
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The next theorem shows that the non-hnearity of middle gates is no problem: 
any such circuit can be transformed into a linear circuit with almost the same 
number of wires. Hence, some linear n-operators require about n^/logn wires 
in such circuits. 

Theorem 2. // a depth-2 circuit computes a linear n-operator and only has 
linear gates on the output layer, then it can be transformed to an equivalent 
linear circuit by adding at most 2n new wires. 

Proof. Let A be an n-by-n (0, l)-matrix, and let $ be a depth-2 circuit com- 
puting Ax. We may assume, for simplicity, that there arc no direct wires from 
inputs to outputs: this can be easily achieved by adding n new wires on the 
first level. Assume that all output gates of $ are linear boolean functions. By 
adding one constant- 1 function on the middle layer and at most n new wires on 
the second level, we can also assume that each output gate computes just the 
sum modulo 2 of its inputs (and not the negation of this sum). 

Let h = (hi, . . . , hr) : {0, 1}" — t- {0, 1}'' be the operator computed by the 
gates on the middle layer. Since AO = and each output gate computes the 
sum modulo 2 of its inputs, we may assume that /i(0) = as well: If hj (0) = 1 
for some j, then replace the function hj by the function h'^ such that h'^iu) = 
and h'j{x) = hj{x) for all a; 7^ 0. 

Let B be the n-by-r adjacency (0, l)-matrix of the bipartite graph formed by 
the wires joining the gates on the middle layer with those on the output layer. 
Then Ax = B ■ h{x) for all x G {0, 1}". Write each vector x = (xi, . . . , a;„) as 
the linear combination x = Y^^=i ^i^i of unit vectors ei, . . . , e„ S {0, 1}", and 
replace the operator h computed on the middle layer by a linear operator 



Hence, h'{x) = x'^M, where M is an n x r matrix with rows h{ei), . . . , h{en). 
Using the linearity of the matrix- vector product, we obtain that (with all sums 



mod 2): 

B ■ h{x) = A - i ^^XicA = ^^XjAe, = ^^XiB ■ h{ei) = B ■ h'{x) . 



Hence, the new (linear) circuit computes Ax as well. It remains to show that 
the number of wires in does not exceed the number of wires in 

The wires on the second level haven't changed at all. To show that the 
number of wires on the first level has not increased as well, let fanout(a;j) be the 
fanout of the ith input node .x^, and fanin(/ij) the fanin of the j'th gate hj on 
the middle layer. Then fanout(xi) = fanin(/ij) is the total number 

L of wires on the first level. We know that h{Q) = 0, that is, /ij(0) = for all 
j = 1, . . . ,r. Now we make a simple (but crucial) observation: if there is no 
wire from Xi to hj, then hj{ei) = hj(0) = 0. This implies that the jth column 
of M can have at most fanin(ft,j) ones. Since the number of wires on the first 
level of is just the total number of I's in M, we are done. □ 
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The second case — when only gates on the middle layer are required to be 
linear — is more delicate. That such circuits can be more powerful than linear 
ones, was shown in [J. Given a boolean n x n matrix A, say that a circuit 
weakly computes the operator /a (a?) = Ax if it correctly computes it on all 
n unit vectors ei,...,e„. Note that, for linear circuits, this is no relaxation: 
such a circuit weakly computes fA iff it correctly computes /a on all inputs. 
Hence, some linear operators cannot be weakly computed by linear depth-2 
circuits using fewer than Q(n^ / \ogn) wires. It is however shown in [7] that the 
situation changes drastically if we only use linear gates on the middle layer but 
allow non-linear gates on the output layer, then any linear n-operator can be 
weakly computed using only O(nlogn) wires. 

Still, using Kolmogorov complexity arguments, we can prove that, for some 
matrices A, such circuits require a quadratic number of wires to compute the 
entire operator Ax. 

Theorem 3. // middle gates are required to be linear, then linear n-operators 
/a with S2(/yl) = f2(n^/logn) exist. 

Proof. We use the Kolmogorov complexity argument known as the incompress- 
ibility argument (see for background). Since we have 2" matrices, some 
matrix A requires bits to describe it. Hence, the linear operator fA{x) = Ax 
cannot be described using fewer than — 0(1) bits, as well. 

Fix an arbitrary depth-2 circuit $ computing fA, and assume that all its 
gates on the middle layer are linear. Let L be the number of wires in $. As 
before, we may assume that there are no direct wires from inputs to outputs. 
Our goal is to show that, using the circuit $, the operator fA can be described 
using 0{L\ogn) bits. This will imply the desired lower bound L = il(n^/logn) 
on the number of wires. 

Let r be the number of nodes on the middle layer of $. Since at these nodes 
only linear functions are computed, the first level (between inputs and middle 
layer) computes some linear operator y = Bx, where B is the r-by-n adjacency 
matrix of the bipartite graph formed by the wires joining the gates on the input 
layer with those on the middle layer. Let also C be the n-by-r adjacency matrix 
of the bipartite graph formed by the wires joining the gates on the middle layer 
with those on the output layer. Hence, L = \B\ + \C\ where \B\ denotes the 
number of Is in B. 

Using these two matrices B and C as well as the fact that the operator 
computed by the circuit $ is linear, we can encode this operator using 0{L log n) 
bits as follows. 

o Since \B\ + \C\ = L, both matrices B and C can be described using 
0{L log n) bits, just by describing the positions of their 1-entries. 

o The ith output gate of $ computes gi{Bx), where gi : {0,1}'' {0,1} 
is some boolean function depending only on rows of B seen by this gate, 
that is, on rows corresponding to the di nodes on the middle layer seen by 
this gate. Let Bi be the di x n submatrix of B formed by these rows. 
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Let lm{Bi) = {Bix: x G {0, 1}"} be the column space of Bi. If this space 
has dimension t then any t hnearly independent columns of B form its 
basis. Take the set B'^ = {ui, . . . ,ut} of the first t linearly independent 
columns of Bi, and call it the first basis of lm{Bi). 

o Encode the behavior of Qi on this basis -B- by the string gi{ui), . . . , gi{ut) 
oi t < di bits. The entire string, for all n output gates gi,...,gn, has 

length at most X]i=i — 

Having this encoding, we can recover the value gi{x) of the iih output gate 
on a given input x G {0,1}" as follows. 

1. Compute Tji = BiX. We can do this since the ith row of C tells us what 
rows of B appear in Bi, and we know the entire matrix B. 

2. Take the first basis B'^ of Im(Bi) and write yi as a linear combination 
Vi — Sfe=i ^kUk of basis vectors over GF{2). 

3. Give Zi = Y^\^i \kgi{uk) mod 2 as an output. We can compute this num- 
ber since we know the values giiui), . . . ,gi{ut). 

Since the circuit computes Ax, the ith output gate must compiitc the scalar 
product {di,x) of input vector x with the ith row di of A. Hence, gi{Bx) = 
{di,x), meaning that gi must be linear on \m{B). Since gi can only see the 
middle gates corresponding to the rows of Bi, this implies that gi must be 
linear also on lm{Bi). Thus, 

t t 
Zi = ^Xkgiiuk) = gi(^^>^kUk'j = giiVi) = 9t{Btx) = gi{Bx) , 
fe=l fe=l 

that is, Zi is a scalar product of x with the ith row of A, as desired. □ 

4 Conclusion 

We have shown that, even when arbitrary boolean hmctions can be used as 
gates, some operators / : {0, 1}" — >■ {0, 1}" require about wires. We have 
also shown that some linear operators require about n^/logn wires in depth-2 
circuits, if either all output gates or all gates on the middle layer are required 
to be linear. 

We conjecture that the same lower bound for depth-2 circuits computing 
linear operators should also hold without any restrictions on used gates. 
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